1. Field of the Invention
The present invention relates to an endoscopic image processing device for estimating a three-dimensional shape of an object by employing correlations between the positions of the same point on a plurality of images of the same object.
2. Description of the Related Art
When the relative arrangement of the imaging means disposed with overlap in a visual field is known, various methods are known which estimate the shape of an object from the images formed, i.e., for estimating the shape from stereo-images.
In recent years, some methods have also been proposed for determining a three-dimensional structure from information about the movement of an object. The methods employ the technique of estimating the relative movement of imaging means from a plurality of images.
All methods use as original information the assumption that a plurality of images at the same point on an object have a positional relation.
Various methods have also been proposed for detecting the positions of the same point on images of an object. The method of extracting a structure by extracting segments is effective for an artificial object because angles and contour components of the object are frequently clear. However, this method cannot be easily applied to a general natural image. Although a density gradient method which is frequently used for time series images produces good results when the apparent movement of an object between images is small and when the image quality is good, imaging conditions are greatly limited.
A block matching method is generally used in which a target point on an image and the circumference thereof are selected as reference areas, and correlation operation is performed between the reference areas and the areas selected from images of a search object to obtain as corresponding points positions having the maximum correlation value. Although the block matching method produces relatively stable results when the image has a clear texture, the method sometimes causes error in detection of a corresponding point when the texture is unclear.
The block matching method also has an intrinsic fault in that when an object is a solid body and contains, for example, a boundary within a block on the background, in which the detection results obtained are unreliable. When the images formed by a plurality of imaging means exhibit large differences in shape and size due to the inclination of the object surfaces with respect to the imaging means or differences in distance between the object and the imaging means, the results obtained are also unreliable.
Problems with respect to the estimation of a three-dimensional shape include problems with respect to occlusion and matching of the estimated results. Particularly, there are portions which cannot be seen from the imaging means side and are not imaged, or portions which are imaged by only one of the imaging means on estimation from stereoscopic images, and handling of these portions is a problem.
When the number of the imaging means is increased, the number of regions to be imaged is inevitably increased, and the number of portions which cannot be seen from the imaging means side is thus decreased. Particularly, when the positions of the imaging means are not known, or when estimation cannot be correctly made, however, the matching between the images is difficult.
Most of the conventional methods have been proposed for images of an artificial object. Alternatively, the effects of the problems inevitably produced when a method is devised for images of a natural object have been removed or decreased by providing some assumptions. Thus it cannot be said that the conventional methods are sufficient for practical use.
Examples of images to which the conventional methods cannot be easily applied and which exhibit a large practical value when being actually employed include images obtained from a bioendoscope.
In an endoscope which permits an affected part in a body cavity to be observed and, if required, the affected part to be treated using a treatment apparatus by inserting a long thin insertion portion into the body cavity without the need for discission, since the size of the tip portion must be minimized in view of the function, it is impossible to incorporate members other than members necessary for a doctor to observe or treat the affected part.
Several proposals have already been made for determining the shape of an object in a trans-endoscopic manner. Examples of proposals include a method of projecting pattern light or the like to a observation object (Japanese Patent Laid-Open No. 64-64525), and a method of providing a compound eye at the tip portion of an endoscope (Japanese Patent Laid-Open No. 63-244011). Since both methods require the tip portion of the endoscope or a light source portion to have a specific structure, the device used is increased in size and complicated, thereby causing difficulties in general use.
Japanese Patent Application No. 62-168791 filed by the same assignee discloses a method of estimating the shape of an object from a plurality of images obtained by moving the tip of an endoscope by manual operation, and a measurement mechanism for measuring the movement amount of the tip moved by the operation. This method can estimate the shape of an object without losing the function of the present endoscope.
However, when this method is used, particularly, for an endoscope used in a living body, the relative movement of the object to the tip of the endoscope, which is caused by a difference in the imaging time phase, cannot be disregarded. In addition, although the precision of the measurement mechanisms is not insufficient in view of the resolution of the imaging means and the like, it is difficult to say that the precision is sufficient.
In addition, since the bioendoscope is mainly used for observing the image of a mucous membrane without clear angles, contours and textures, the detection of corresponding points remarkably causes problems, and it is necessary to take some measures against the problems of the detection results of the corresponding points employed for estimation.
As described above, there is no method or device for estimating the shape of an object which is capable of stably producing good results of estimation of the positions of imaging means and the shape of the object even if applied to a natural image having no clear characteristic point, for example, an endoscopic image, and which has no need for changing the device (increasing the size of the device or the like) to an extent which affects the intrinsic observation function.